Investigation of similarity-based test case selection for specification-based regression testing.

uring software maintenance, several modifications can be performed in a specification model in order to satisfy new requirements. Perform regression testing on modified software is known to be a costly and laborious task. Test case selection, test case prioritization, test suite minimisation,among other methods,aim to reduce these costs by selecting or prioritizing a subset of test cases so that less time, effort and thus money are involved in performing regression testing. In this doctorate research, we explore the general problem of automatically selecting test cases in a model-based testing (MBT) process where specification models were modified. Our technique, named Similarity Approach for Regression Testing (SART), selects subset of test cases traversing modified regions of a software system’s specification model. That strategy relies on similarity-based test case selection where similarities between test cases from different software versions are analysed to identify modified elements in a model. In addition, we propose an evaluation approach named Search Based Model Generation for Technology Evaluation (SBMTE) that is based on stochastic model generation and search-based techniques to generate large samples of realistic models to allow experiments with model-based techniques. Based on SBMTE,researchers are able to develop model generator tools to create a space of models based on statistics from real industrial models, and eventually generate samples from that space in order to perform experiments. Here we developed a generator to create instances of Annotated Labelled Transitions Systems (ALTS), to be used as input for our MBT process and then perform an experiment with SART.In this experiment, we were able to conclude that SART’s percentage of test suite size reduction is robust and able to select a sub set with an average of 92% less test cases, while ensuring coverage of all model modification and revealing defects linked to model modifications. Both SART and our experiment are executable through the LTS-BT tool, enabling researchers to use our selections trategy andr eproduce our experiment.During software maintenance, several modifications can be performed in a specification model in order to satisfy new requirements. Perform regression testing on modified software is known to be a costly and laborious task. Test case selection, test case prioritization, test suite minimisation,among other methods,aim to reduce these costs by selecting or prioritizing a subset of test cases so that less time, effort and thus money are involved in performing regression testing. In this doctorate research, we explore the general problem of automatically selecting test cases in a model-based testing (MBT) process where specification models were modified. Our technique, named Similarity Approach for Regression Testing (SART), selects subset of test cases traversing modified regions of a software system’s specification model. That strategy relies on similarity-based test case selection where similarities between test cases from different software versions are analysed to identify modified elements in a model. In addition, we propose an evaluation approach named Search Based Model Generation for Technology Evaluation (SBMTE) that is based on stochastic model generation and search-based techniques to generate large samples of realistic models to allow experiments with model-based techniques. Based on SBMTE,researchers are able to develop model generator tools to create a space of models based on statistics from real industrial models, and eventually generate samples from that space in order to perform experiments. Here we developed a generator to create instances of Annotated Labelled Transitions Systems (ALTS), to be used as input for our MBT process and then perform an experiment with SART.In this experiment, we were able to conclude that SART’s percentage of test suite size reduction is robust and able to select a sub set with an average of 92% less test cases, while ensuring coverage of all model modification and revealing defects linked to model modifications. Both SART and our experiment are executable through the LTS-BT tool, enabling researchers to use our selections trategy andr eproduce our experiment

Visualizing test diversity to support test optimisation

Diversity has been used as an effective criteria to optimise test suites for cost-effective testing. Particularly, diversity-based (alternatively referred to as similarity-based) techniques have the benefit of being generic and applicable across different Systems Under Test (SUT), and have been used to automatically select or prioritise large sets of test cases. However, it is a challenge to feedback diversity information to developers and testers since results are typically many-dimensional. Furthermore, the generality of diversity-based approaches makes it harder to choose when and where to apply them. In this paper we address these challenges by investigating: i) what are the trade-off in using different sources of diversity (e.g., diversity of test requirements or test scripts) to optimise large test suites, and ii) how visualisation of test diversity data can assist testers for test optimisation and improvement. We perform a case study on three industrial projects and present quantitative results on the fault detection capabilities and redundancy levels of different sets of test cases. Our key result is that test similarity maps, based on pair-wise diversity calculations, helped industrial practitioners identify issues with their test repositories and decide on actions to improve. We conclude that the visualisation of diversity information can assist testers in their maintenance and optimisation activities

A Testability Analysis Framework for Non-Functional Properties

This paper presents background, the basic steps and an example for a testability analysis framework for non-functional properties

Boundary Value Exploration for Software Analysis

For software to be reliable and resilient, it is widely accepted that tests must be created and maintained alongside the software itself. One safeguard from vulnerabilities and failures in code is to ensure correct behavior on the boundaries between sub-domains of the input space. So-called boundary value analysis (BVA) and boundary value testing (BVT) techniques aim to exercise those boundaries and increase test effectiveness. However, the concepts of BVA and BVT themselves are not clearly defined and it is not clear how to identify relevant sub-domains, and thus the boundaries delineating them, given a specification. This has limited adoption and hindered automation. We clarify BVA and BVT and introduce Boundary Value Exploration (BVE) to describe techniques that support them by helping to detect and identify boundary inputs. Additionally, we propose two concrete BVE techniques based on information-theoretic distance functions: (i) an algorithm for boundary detection and (ii) the usage of software visualization to explore the behavior of the software under test and identify its boundary behavior. As an initial evaluation, we apply these techniques on a much used and well-tested date handling library. Our results reveal questionable behavior at boundaries highlighted by our techniques. In conclusion, we argue that the boundary value exploration that our techniques enable is a step towards automated boundary value analysis and testing which can foster their wider use and improve test effectiveness and efficiency

Análise das relações sociais e econômicas no comércio atacadista de pescado fresco no mercado do Ver-o-Peso, em Belém-Pará

Dissertação (mestrado)—Universidade de Brasília, Faculdade de Agronomia e Medicina Veterinária, Programa de Pós-Graduação em Agronegócios, 2014.O Mercado do Ver-o-Peso em Belém do Pará é um dos principais pontos de desembarque e comercialização de pescado do Brasil. Por ele desembarcam 35 mil toneladas de pescado por ano para o abastecimento da Região Metropolitana de Belém e o envolvimento de centenas de agentes, dentre os quais se destacam os balanceiros, agentes comissionados que comercializam a produção dos pescadores no local. Há tempos idealizada e agora em execução, a construção do Terminal Pesqueiro Público de Belém, financiada pelo Ministério da Pesca e Aquicultura representa um risco ao modelo de comercialização e distribuição de pescado atual, construído socialmente ao longo de três séculos. O presente trabalho tem o intuito de demonstrar, através dos pressupostos da Sociologia Econômica, a importância dos balanceiros no processo de comercialização e distribuição do pescado no Ver-o-Peso. Como metodologia utilizada para coleta de dados foram realizadas revisões bibliográficas e documentais, entrevistas semiestruturas, observação diretas e registros áudio visuais. A análise de conteúdo foi feita a partir da tabulação e classificação dos dados de acordo com os pressupostos da Sociologia Econômica. Os resultados apresentados demonstram a importância do balanceiro como coordenador logístico das transações comerciais realizadas no comércio atacadista de pescado fresco na Pedra do Ver-o-Peso. Que este trabalho permita um olhar diferenciado aos acadêmicos do setor através de uma perspectiva baseada nas motivações individuais e coletivas, nas relações estabelecidas entre os agentes e na construção social dos mercados, contexto que extrapola o pressuposto econômico da oferta e demanda, onde a palavra-chave é a IMERSÃO SOCIAL.The “Ver-o-Peso” Market, freely translated as “check-the-weight” market, is one of the main fish landing and commercialization sites in Brazil. Thirty five tons of fish land yearly on the market with the involvement of several groups of individuals, among which the “balanceiros,” local conditioned agents that commercialize the local catch, deserve special highlight. The eminent conclusion of the construction of the Public Fishing Port of Belem represents a risk to current fish the commercialization model, socially built thought three centuries. The present research pretends demonstrate, though Economical Sociology assumptions, the importance of the “balanceiros” in the fish commercialization model installed in the “Ver-o-Peso” Market. The adopted methodology included bibliographic research, documental research, semi structured interviews, direct observation audio registration and video registration. The analysis of the collected information was conducted through tabulation and ranking according to the assumptions of the Economical Sociology. The results demonstrate the importance of the “balanceiro” as the logistical commerce coordinator of the transactions realized in the wholesale market of fresh fish on the “pedra,” commercialization open slots on the “Ver-o-Peso” Market. This study offers sector researchers a differentiated perspective based on individual and collective motivations of the individuals involved in the established and developing local markets, in a perspective that extrapolates the economical assumptions of supply and demand, in a perspective that the keyword are social immersion or embeddeness

Evolution of statistical analysis in empirical software engineering research: Current state and steps forward

Software engineering research is evolving and papers are increasingly based on empirical data from a multitude of sources, using statistical tests to determine if and to what degree empirical evidence supports their hypotheses. To investigate the practices and trends of statistical analysis in empirical software engineering (ESE), this paper presents a review of a large pool of papers from top-ranked software engineering journals. First, we manually reviewed 161 papers and in the second phase of our method, we conducted a more extensive semi-automatic classification of papers spanning the years 2001--2015 and 5,196 papers. Results from both review steps was used to: i) identify and analyze the predominant practices in ESE (e.g., using t-test or ANOVA), as well as relevant trends in usage of specific statistical methods (e.g., nonparametric tests and effect size measures) and, ii) develop a conceptual model for a statistical analysis workflow with suggestions on how to apply different statistical methods as well as guidelines to avoid pitfalls. Lastly, we confirm existing claims that current ESE practices lack a standard to report practical significance of results. We illustrate how practical significance can be discussed in terms of both the statistical analysis and in the practitioner's context.Comment: journal submission, 34 pages, 8 figure

Challenges and guidelines on designing test cases for test bots

Test bots are automated testing tools that autonomously and periodically run a set of test cases that check whether the system under test meets the requirements set forth by the customer. The automation decreases the amount of time a development team spends on testing. As development projects become larger, it is important to focus on improving the test bots by designing more effective test cases because otherwise time and usage costs can increase greatly and misleading conclusions from test results might be drawn, such as false positives in the test execution. However, literature currently lacks insights on how test case design affects the effectiveness of test bots. This paper uses a case study approach to investigate those effects by identifying challenges in designing tests for test bots. Our results include guidelines for test design schema for such bots that support practitioners in overcoming the challenges mentioned by participants during our study.Comment: To be published in IEEE/ACM 42nd International Conference on Software Engineering Workshops (ICSEW'20), May 23--29, 2020, Seoul, Republic of Kore

Validation of the ISAREG model for the irrigation management of the melon crop in the state of Ceará

The research aimed to validate the ISAREG model, introducing it to water management studies in irrigation in the State of Ceará, comparing results of experimental research, with results of simulations, carried out with the use of software, analyzing the following variables: crop evapotranspiration, variation of soil water storage and water flow in the soil. A bibliographic survey was carried out to obtain soil, climate and crop information required by the model to perform the soil water balance. Aiming at the validation of ISAREG, the model was fed with the following data: reference evapotranspiration, precipitation, phenological phases of the crop, effective depth of the root system, water availability factor in the soil, crop coefficient and soil information. Subsequently, the irrigation management option "dates and irrigation depths" was selected, and ISAREG performed the simulation of the soil water balance. The ISAREG demonstrated a detailed soil water balance, being validated in this study, because when its results were compared to the experimental ones, there was similarity in the trends of the variables analyzed, despite the reduced correlation verified regarding the variation of water storage in the soil

Avaliação de argamassas com incorporação de resíduos de britagem para construções rurais

The exploitation of mineral reserves in order to produce aggregate for making mortar and concrete has mortar has seen high growth in the northeast of the country, especially in Paraiba state. However, this operation entails the production of waste from the process of crushing rocks to reach the fraction of aggregate trade, known as tailings, thus polluting the environment. This work aimed to evaluate the compressive strength of mortars containing different amounts of tailings instead of natural sand for rural construction. The procedures according to NBR NM 248 (selection and classification of the tailings) and the NBR 13279 (determination of the uniaxial compressive strength) were performed. The results showed that the highest strength for the mortars with higher level of tailings and water/cement ratio was obtained by using 22 MPa for water/cement ration equal 0.65 and 100% replacement content of sand by tailings.A exploração de reservas minerais com a finalidade de produção de agregados para confecção de argamassas e concretos tem tido alto crescimento na região nordeste do país, sobretudo na Paraíba. Entretanto, essa exploração traz consigo a produção de rejeitos oriundos do processo de britagem de rochas maiores até atingir a fração comercial de agregado, denominados de resíduos de britagem tornando-se, portanto, poluidores do ambiente. Este trabalho teve como objetivo avaliar a resistência à compressão de argamassas com diferentes teores de resíduos de britagem em substituição da areia natural para construções rurais. Os procedimentos foram realizados de acordo com a NBR NM 248 (seleção e classificação do resíduo) e com a NBR 13279 (determinação da resistência à compressão uniaxial). Os resultados mostraram que a mais alta resistência foi obtida para a argamassa com maior teor de resíduo e relação água/cimento, ou seja, 22 MPa para a relação água/cimento 0,65 com um teor de incorporação de 100%

A Method to Assess and Argue for Practical Significance in Software Engineering

A key goal of empirical research in software engineering is to assess practical significance, which answers whether the observed effects of some compared treatments show a relevant difference in practice in realistic scenarios. Even though plenty of standard techniques exist to assess statistical significance, connecting it to practical significance is not straightforward or routinely done; indeed, only a few empirical studies in software engineering assess practical significance in a principled and systematic way. In this paper, we argue that Bayesian data analysis provides suitable tools to assess practical significance rigorously. We demonstrate our claims in a case study comparing different test techniques. The case study\u27s data was previously analyzed (Afzal et al., 2015) using standard techniques focusing on statistical significance. Here, we build a multilevel model of the same data, which we fit and validate using Bayesian techniques. Our method is to apply cumulative prospect theory on top of the statistical model to quantitatively connect our statistical analysis output to a practically meaningful context. This is then the basis both for assessing and arguing for practical significance. Our study demonstrates that Bayesian analysis provides a technically rigorous yet practical framework for empirical software engineering. A substantial side effect is that any uncertainty in the underlying data will be propagated through the statistical model, and its effects on practical significance are made clear. Thus, in combination with cumulative prospect theory, Bayesian analysis supports seamlessly assessing practical significance in an empirical software engineering context, thus potentially clarifying and extending the relevance of research for practitioners